Visual Argument Structure Tool (VAST) by Leising, Grenke & Cramer
2024-11-08
Tip
The full list of formalization steps is presented on this website. This slide set covers steps 2 to 6 of the broader scheme.
Formalization approaches can differ in their starting point (which often are not clear-cut distinct):
Most theories in psychology are too fuzzy and broad to be formalized in one round. We restrict the scope of the theory to keep modelling feasible:
This table will be called the Construct Source Table, as it collects the original sources for the definitions of the constructs.
| ID | Type | Short name | Quote | Reference | rel. type (n, p, i, r, …) | Comment | Incl. (Y/N) |
|---|---|---|---|---|---|---|---|
| A | P | bystander effect | “The bystander effect refers to the phenomenon that an individual’s likelihood of helping decreases when passive bystanders are present in a critical situation.” | Fischer et al. (2011), p. 517 | n, p | Y | |
| B | |||||||
| … |
n) relationship for each construct.Robustness of phenomena has two dimensions:
Practically, you should do the following steps to assess these two dimensions:
When you assess the generalizability, you should distinguish three prototypical epistemic states:
When is the strength of evidence strong? If …
As a sidenote: Ideally, evidence is quantified with a statistical technique that also allows to measure evidence for the null hypothesis and that gives a continuous quantification of the strength of evidence. A Bayes factor provides both desiderata.
Note
General principle: We can only make statements about stuff that we actually studied.
We describe six prototypical examples:
All four UTOS dimensions can get an independent assessment. Consider the ManyLabs2 study, where identical experiment (except translation of materials) has been administered online in very diverse samples (at least diverse with respect to nationality and cultural background).
Tip
There was high variation in the U(nits) and the S(ettings) dimensions, but very low variation in the T(reatment) and O(utcome) dimension.
Non-zero effects could be found with remarkably low variability across samples: “Cumulatively, variability in the observed effect sizes was attributable more to the effect being studied than to the sample or setting in which it was studied” (quoted from the Abstract).
Figure 2: A generalizability assessment of the UTOS dimensions in ManyLabs 2.
Hence, we have strong evidence for high generalizability for the U and the S dimension, but we cannot make a conclusion concerning the T and the O dimensions, as they lacked the necessary variation in the study.
“A meta-analysis of 78 studies underlines that social loafing is a robust phenomenon that generalizes across tasks and populations (Karau & Williams, 1993).
Regarding different units, social loating was found among groups of organizational employees, college students as well as high school students. Furthermore, the phenomenon appears within different genders as well as various cultures.
The studies mentioned in the meta-analysis used a variety of different tasks as treatment, such as physical exercises (e.g., rope pulling), cognitive tasks (e.g., brainstorming), evaluative efforts (e.g., rating items) and perceptual tasks (e.g., monitoring a screen for signals). These tasks also differed in complexity.
Regarding the outcomes, social loafing was generally assessed by comparing individual’s coactive efforts with the individual’s collective efforts. The specific units of measurement varied across tasks, with some studies focusing on quantity of individual contributions and others taking the quality into account.
Lastly, concerning different settings of studies, the meta analysis included laboratory studies as well as field studies.
In sum, we can conclude that evidence for social loafing found in this meta-analytic review is strong and the phenomenon generalizes over all of the UTOS dimensions. Social Loafing can be consider to be a robust phenomenon.”
These slides are part of the course Formal modeling in psychology at LMU Munich